Machine learning can be employed to construct a model or rule set to predict a result based on values with respect to a number of features. A series of input patterns can be provided to an algorithm along with a desired output (e.g., the label) and the algorithm then learns how to classify the patterns by outing a desired label. In supervised learning (e.g., Kernal-based support vector machine (SVM) algorithm), a human operator must provide the labels during a teaching phase. Alternatively, unsupervised clustering is a process of assigning labels to the input patterns without the use of the human operator. Such unsupervised methods generally function through a statistical analysis of the input data by determining an Eigen value vector of a covariance matrix.
The majority of prior art machine learning approaches utilize many patterns or exemplars for learning. The variables that encode the algorithms learning behavior can be modified during a learning stage and tested for accuracy and generalization during a testing phase. Without sufficient examples, determining a solution that balances memorization with generalization is often difficult due to separation of the training and testing stage. Also, it is difficult or impossible to determine an appropriate variable configuration leading to an optimal point during the learning stage.
A modern solution to the memorization vs. generalization problem involves the mathematical technique of support-vector-maximization. The input patterns can be projected into a high-dimensional and linearly separable space and a linear classifier can then be employed to label the data in binary classification. The linear classifier represents a hyperplane (e.g., a decision boundary) in a high-dimensional space. All inputs falling on one side of the decision boundary result in a positive output, while all inputs on the other side result in a negative output. The support-vectors are the distances from the closest input points to the decision boundary and the process of maximizing the distance is support-vector-maximization. The problem associated with such an approach is that identifying the support-vectors without sufficient examples requires extensive testing of a number of input patterns to determine which input is closest to the decision boundary.
Another problem associated with machine learning is adaptation to non-stationary statistics, which can occur as the statistic of the underlying data varies with time. Also, determining statistical regularities in large quantities of streaming information can be incredibly power intensive as the problem encounters combinatorial explosions. The complexity of the task is echoed in a biological nervous system, which are essential communication networks that self-evolve to detect and act on regularities present in the input data stream.
Based on the foregoing, it is believed that a need exists for an improved machine learning system and method. A need also exists for an improved method for extracting feature with respect to an input data stream, as described in greater detail herein.